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1. INTRODUCTION 

Smartphones are becoming an important part of our lives. The comfort and convenience provided by 
smartphones are certainly improving our lives more than ever before. Currently, smartphone features include 
digital cameras and the internet, which we can access whenever and wherever we need it. Therefore, it has 
become possible to access the internet, find information pointed by a camera embedded in a smartphone, and 
provide it by voice. The spread of internet use is easy to connect to web pages using PCs, personal digital 
assistants (PDAs), or mobile phones over a wired or wireless network. In particular, it has brought a lot of 
convenience to everyday life for users who use smartphones to search for web pages. As users are aware, using 
computer keyboards and mobile phone keypads brings much more inconvenience and difficulty compared to 
the time spent connecting to a web page between mobile phone keypads for entering uniform resource locator 
(URL). To solve this problem, quick response (QR) codes have emerged. QR code means that a quick response 
can be obtained. 

The purpose of the initial QR code is to use URL information converted into a pattern of QR codes 
and a quick connection to related web pages. The biggest advantage of QR codes is that they can hold a large 
amount of data and information compared to traditional bar codes, while keeping the code size short and small. 
In addition, QR codes have an error recovery function, which allows data information to be restored even if 
parts of the code are contaminated or damaged. The area where QR codes are most actively applied is the 
advertising industry. When sufficient information cannot be delivered due to space constraints, whether online 
or offline, additional information can be referred to on the internet through QR codes. In some cases, office 
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workers add QR codes to their business cards. This is because you can put detailed introductions of the 
company or yourself, personal blog addresses, Facebook accounts, various phone numbers, and photos. There 
is also a genuine authentication system using QR codes. For example, in the field of agricultural, livestock and 
livestock products, QR codes are attached to special products of local governments. The user may scan the QR 
code to obtain producer, product information, quality certification content, and the like for the corresponding 
product. In addition, QR codes are now deeply embedded in our daily lives [1]-[14]. A recent example is 
content that has QR codes attached to paper books such as novels. When this QR code is read, music or video 
related to the content of the text is played. 

As QR codes can be freely produced and used by anyone, there are potential risk factors. Since it can 
contain more information than existing barcodes, it can spread computer malicious codes and harmful website 
addresses as much as possible by QR codes. Instead of blindly scanning with a smartphone because you can 
see the QR code, you should check for what purpose and where it is provided. QR codes are highly utilized 
compared to barcodes, but surprisingly few people use QR codes directly. Because it lacks convenience. 
Moreover, the advantage of smartphones has been that recognition technology using QR codes has been 
developed. However, the use of existing and 2D barcodes in the field of the visually impaired has not gained 
much power compared to other fields of study. Therefore, in this paper, we propose voice processing and image 
processing techniques for images using cameras to support object identification to visually impaired people. 


2. IMAGE SCAN 

Before using an application that uses a camera on a device, consideration should be given to how the 
application wants to use hardware features. This paper considered the requirements for the use of cameras in 
applications, if using existing camera applications, if developing custom camera functions, whether to share 
for use by other applications, and whether to enable the use of photos and videos even when uninstalling 
applications [15]-[18]. With the Android.hardware.camera2 application programming interface and camera 
Intent, the framework of Android enables capturing images or video. Table 1 shows the associated class. 


Table 1. Associated class 


Class Function 
android.hardware .camera2 The first application programming interface for taking command of camera module 
Camera The older disapproval application programming interface for taking command of camera module 
Surface View The camera preview presentation in real time 
Intent Capturing images and videos 


2.1. Manifest declaration 

Before starting application development with camera application programming interfaces, appropriate 
manifest declarations should be done to make it possible to activate the camera module and related functions. 
With this reason, the application requested permission to use the device camera. In the application, the camera 
function is declared to be used, so that the application will not be able to be installed on the equipment without 
cameras or do not support the camera function specified in Google Play. The manifest declared authorization for 
external storage because the application required the image or video to be stored in the external storage of the 
device. To record video or audio, the application should ask audio capture privileges. When an application tags 
an image with location information of global positioning system, it should get admission of 
Access_Fine_Location. If the application is Android 4.0 (application programming interface level 15), the 
application must declare the use of the device's global positioning system. 


2.2. Camera interface custom 

Typical steps to custom a camera interface for an application are camera detection, access, preview 
classes, preview layout creation, capture listener settings, file capture and save, and camera release. In the 
process of detecting and accessing the camera, a code was created to check if there was a camera and request 
access. We then expanded Surface View and created a camera preview class that carries out the interface of 
SurfaceHolder. 

The camera preview class may preview a live image of the camera. When the camera preview class is 
created, a design of view was created that integrates the desired development environment control. In order for 
the capture to operate, after connecting the listener of the image control using a button, a code for storing the 
output of the captured picture was set. After using the camera, the application was appropriately released for 
use by other applications. 
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3. IMAGE ANALYSIS AND VOICE OUTPUT 
3.1. Bitmap image load 

To retrieve the stored file image, the absolute path was determined using the getAbsolutePath() 
method to know the path of the file returned by getOutputMediaFile(). An absolute path search is an absolute 
path to external memory if there is external memory and transmits the internal memory location when there is 
no external memory. In order to process an image, a file must be received as a bitmap image, so a Bitmap 
object was created, which reads the path where the file is located and contains a decoded bitmap image. 
Figure | shows the structure of the bitmap image. The bitmap class is used to show image files on the canvas, 
and the image is displayed in the drawable folder or the onDraw() method showing the image file on the SD 
card is used to verify that the image is invoked [19]-[22]. 


Bitmap file header Very simple header info 


Detailed info about data 
and BMP dimensions 


Palette if palletized RGB quad[] 


RGB pixels Actual data for bitmap 


or 
Indexed data 


Figure 1. Bitmap structure 


3.2. Color recognition 

Each pixel of the bitmap image represents a specific color. Bitmap images have three bytes of one 
pixel of color and are included in RGB order. If there is a bitmap image 100 px by 100 px, RGB information 
of 100 X 100 = 10,000 and 10,000 x3 = 30,000 Bytes is stored. In addition, it should be noted that the bitmap 
image is a coordinate (width, height) from the upper-left coordinate (0,0) to the lower-right (width, height), so 
when storing it in a two-dimensional array, x and y coordinates must be stored interchangeably. Color 
information of one pixel is stored in byte form, stored in eight hexadecimal digits, and is determined by the 
ratio of R, G, and B colors, excluding the previous two digits. It can be seen that these images consist of three 
matrices as shown in Figure 2. Each pixel value of the square is also shown in Figure 2. Figure 2(a) shows the 
image, and detail values of red, green, blue pixels are as shown in Figures 2(b)-(d) respectively. 

In order to recognize the color, you need to know the ratio of R, G, and B values. A certain part of the 
center of the camera was used as a range, and pixel values of coordinates were received using the getPixel() 
method. In order to extract R, G, and B values from pixel values, bit operations have been performed to read 
the values of each digit [23]. In order to recognize it more efficiently, the color of clothes that people usually 
wear and the Android color table were compared and analyzed. 


(d) 


198 


(b) 
Figure 2. Color image: (a) image, (b) red pixels, (c) green pixels, and (d) blue pixels 


3.3. Voice output 

The media player class which is provided to operate multimedia functions provides music and video. 
The play(), pause(), and stop() methods of the media player class have the ability to start, pause, and stop music 
and videos, respectively. Before outputting music, the mp3 file required for voice output was placed in the raw 
folder. If the conditions for color recognition were satisfied, a voice file was output [24]-[28]. 


Security system using mobile image processing and color recognition for the visually ... (Eugene Rhee) 


1366 O ISSN: 2502-4752 


3.4. Image binarization 

Binary image refers to an image that expresses all pixels only in black and white. It is similar to the 
grayscale image, but different from the grayscale image. The grayscale image is an image that expresses the 
brightness of step 256 and the binarized image is a black and white image consisting only of 0 and 1. Binaryization 
of an image is a process of making a pixel having a low value based on a boundary value of 0 and a pixel having 
a high value of 255. In this way, the binarized image is used to detect the characteristics of an object included in 
the image and is known as a basic work for use in image processing applications such as license plate recognition 
[29]-[40]. Using the binarized image, only the letter part can be clearly characterized and shown in the image. In 
addition, the size of the image file decreases through the binarization process [29]-[40]. 


4. RESULTS AND DISCUSSION 

In this paper, a biometrics recognition technique for improving the convenience of the visually 
impaired is proposed. In this regard, the user's efficiency and independence are verified through a color 
recognition application based on the Android camera function in the user's daily life. This application consists 
of a camera preview function, a color recognition function, and a function of informing color information 
through voice processing. 


4.1. Camera preview 

The camera preview function implements the function of displaying the camera screen on the smart 
device on the frame layout screen of the application to specify the color that the user wants to recognize. When 
user runs the application, a screen that can be captured appears. When the capture button is pressed, the capture 
screen is stored under the file name IMG_captured time.jpg in the MyCameraApp folder in the external or 
internal repository, just as the camera application stores the photo in the album. 


4.2. Representative color setting of captured image 

Image color recognition physically recognizes the color of the image through the Android color table. 
Existing applications had to have an intermediate medium called QR code directly by the user [49], but in this 
paper, the application does not need a QR code, directly recognizes it, and outputs it as a voice. In order to 
recognize colors, the captured image is stored in the bitmap image variable to receive pixel values, and the color 
that was most recognized as the same color within the error range is output as a representative color. 


4.3. Voice output of color information 

If the image displayed on the screen is saved through the capture button of the application and the color 
of the image is recognized, the mp3 file stored in the raw folder file is found and executed. The user may recognize 
the color by voice by pressing the button at a location where the color desired to be known is located. RGB values 
and colors were output to TextView to show the process of color recognition and output as a voice. 


4.4. Image pattern recognition 

Most clothes are often monochrome without patterns. In the case of patterned clothes, the color based 
on was extracted, and the color extracted through binary image was expressed in white to confirm the pattern. 
Usually, in the case of binarization of an image, the average value of the pixel is extracted and expressed in 
white if the pixel value is greater than the average value, and in black if the pixel value is less than the average 
value. However, the purpose of binarization in this paper is to determine the distribution of recognized colors, 
so the pixel values within the error range of the extracted colors were set to white and all others to black. In 
this paper, ImageView is made and showed a binarized video. 


5. CONCLUSION 

In our daily lives in reality, we can provide accurate information to application users in a better way 
through virtual space or any object in the computer. In this paper, we analyze the color of clothes that people 
often wear and the RGB color table of Android for color recognition, which can provide the convenience of 
daily life for the blind using mobile cameras and image processing processes. First of all, the camera illuminates 
the place you want to recognize by voice. It is a system that recognizes a certain part of the color and outputs 
the underlying color through voice when the capture button is pressed after checking the camera preview screen 
of the application. In other words, it is a color recognition voice system, and in this paper, for a visually 
identifiable effect, it is possible to check the position of the representative color captured by the output of the 
image through binarization of the image. The application operation process is described in detail and confirmed 
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through actual experiments. In the future, research should be conducted not only to improve color recognition 
due to light reflection, but also to effectively recognize numbers and letters and quickly transmit them to voice. 
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